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Mr,  Chairman  and  Members  of  the  Subcommittee: 


Thank  you  for  the  opportunity  to  participate  in  this  important 
discussion  about  one  of  the  most  critical  Department  of  Defense 
information  system  initiatives,  the  Standard  Procurement  System 
(SPS)  . 


As  requested  in  your  invitation  letter,  my  statement  focuses 
primarily  on  the  results  of  my  office's  three  audits  on  this 
program.  First,  to  put  our  findings  on  SPS  into  a  broader 
context,  I  will  repeat  a  few  observations  from  my  testimony 
before  you  last  March  on  Top  Defense  Management  Challenges. 


"Given  the  considerable  dependence  on  "IT"  and  the  high 
cost  of  large  system  investments,  the  historically  poor 
record  of  the  DoD  for  controlling  the  proliferation  of 
incompatible  systems  with  nonstandard  data  elements, 
acquiring  new  systems  that  meet  user  needs  within 
reasonable  timeframes,  controlling  cost,  and  ensuring  the 
quality  and  security  of  data  has  been  a  major  concern. 
Recognizing  that  such  problems  are  common  across  the 
Federal  Government,  the  Congress  specified  in  the  Clinger- 
Cohen  Act  of  1996  that  Chief  Information  Officers  in  each 
agency  would  oversee  well  disciplined  information 
technology  acquisition  processes.  This  is  a  daunting 
challenge  for  a  department  with  71  major  information  system 
acquisition  projects  and  hundreds  of  "smaller"  system 
acquisition  and  modification  projects  belonging  to  dozens 
of  organizations.  The  DoD  has  been  candid  about  the  need 
for  more  effective  management  controls  in  this  crucial 
area,  but  progress  has  been  slow  and  the  goals  of  the 
dinger- Cohen  Act  have  not  yet  been  achieved. 
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The  Department  has  revised  its  basic  information  system 
acquisition  procedures  and  tried  to  be  responsive  to  our 
recommendations.  Nevertheless,  we  believe  this  area 
deserves  continued  close  attention. ...  At  the  present  time, 
virtually  every  information  technology  project  that  we 
audit  exhibits  significant  management  problems.  Those 
flaws  include  poorly  defined  requirements  and  frequent  user 
dissatisfaction. " 


The  SPS  is  one  of  those  71  major  systems.  Many  of  the  issues 
raised  during  the  past  few  years  by  my  office,  SPS  users  and  the 
General  Accounting  Office  pertained  directly  to  incomplete 
compliance  with  the  Clinger-Cohen  Act,  inadequate  definition  of 
requirements  and  user  dissatisfaction.  Its  problems  and 
challenges  are  not  atypical  in  the  DoD  information  system 
acquisition  world. 


When  discussing  the  SPS  program,  it  is  also  important  to  keep  in 
mind  that  it  represents  one  of  the  Department ' s  most  aggressive 
attempts  to  modernize  and  standardize  its  information  processing 
in  a  core  business  function.  The  SPS  was  originally  intended  to 
replace  a  jumble  of  between  70  and  80  existing  systems  and  to 
support  what  the  Department  refers  to  as  the  End-to-End 
Procurement -Finance  Model.  The  concept  of  a  common  system  to  be 
used  by  all  Military  Departments  and  Defense  Agencies  for  the 
whole  gamut  of  procurement  tasks,  with  efficient  links  to  the 
finance  systems,  is  extremely  challenginof .  Nevertheless,  it  has 
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been  well  worth  pursuing  even  if  total  standardization  proves  to 
be  impossible,  as  is  likely  to  be  the  case. 

Finally,  the  SPS  program  merits  close  attention  because  of: 

-  the  need  for  productivity  enhancing  information  tools  to 
offset  the  50  percent  reduction  of  the  Defense  acquisition 
workforce  over  the  past  several  years; 

-  its  prominence  as  a  major  attempt  to  tailor  off-the-shelf 
commercial  software  for  government  use;  and 

-  its  spiral  development  and  incremental  deployment 
approaches,  which  have  been  widely  touted  as  the  best  means  to 
get  new  technology  to  users  as  quickly  as  possible. 

Now  to  recap  our  three  reports  on  SPS. 

The  SPS  program  began  in  November  1994.  From  April  through 
October  1995,  four  draft  contract  solicitations  were  issued  for 
comment.  During  that  process,  we  received  allegations  to  the 
DoD  Hotline  regarding  what  functional  requirements  had  been 
initially  identified,  the  program's  acquisition  approach  and  its 
testing  plans.  We  reported  our  findings  on  these  matters  in 
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Audit  Report  No.  96-219,  Allegations  to  the  Defense  Hotline 
Concerning  the  Standard  Procurement  System,  September  5,  1996. 

We  determined  that,  despite  Assistant  Secretary  of  Defense 
(Command,  Control,  Communications  and  Intelligence)  review  and 
approval  for  SPS  to  proceed  past  Acquisition  Milestone  I  in 
August  1995,  much  more  needed  to  be  done  to  control  program 
risks.  Specific  risks  were  as  follows: 

-  the  testing  plan  was  inadequate  because  of  uncertain 
operational  performance  requirements  and  a  compressed  schedule; 

-  the  acquisition  strategy  was  to  purchase  an  off-the-shelf 
commercial  system,  although  it  was  recognized  that  very 
substantial  follow-on  software  development  would  be  needed  to 
operate  in  the  Defense  environment; 

-  initially,  a  fixed  price  contract  was  planned,  despite 
numerous  uncertainties; 

-  functional  requirements  were  delineated  too  broadly  and 
there  was  insufficient  assurance  that  user  needs  had  been  fully 


identified; 
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-  specific  site  requirements  were  not  well  defined;  and 

-  alternative  deployment  approaches  were  not  fully 
analyzed. 

In  response  to  our  audit,  managers  generally  agreed  that  the 
program  carried  risk  and  took  various  measures  to  formalize  the 
testing  requirements,  change  the  contract  terms  and  provide 
ongoing  monitoring.  We  were  assured  that  each  future 
Acquisition  Milestone  decision  would  be  accompanied  by  a 
rigorous  Office  of  the  Secretary  of  Defense  level  review  of 
system  functionality,  testing  and  risks.  In  retrospect,  it  is 
clear  that  not  enough  was  done  to  keep  the  commitment  expressed 
by  the  Defense  Logistics  Agency  in  reply  to  our  report: 

"By  prior  direction  of  the  Director,  Defense  Procurement, 
SPS  will  not  be  deployed  to  any  DoD  procurement  site 
wherein  we  cannot  provide  equal  to  or  better  than  existing 
functionality. " 

We  performed  a  follow-up  audit  during  1998,  resulting  in  Audit 
Report  No.  99-166,  Initial  Implementation  of  the  Standard 
Procurement  System,  May  26,  1999.  We  raised  or  reiterated  the 
following  concerns: 
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-  The  commercial  software  was  providing  only  45  percent  of 
the  required  functionality,  not  the  60  to  75  percent  that  was 
originally  predicted; 

-  the  drastic  makeover  of  the  system  to  add  functionality 
had  created  a  DoD-unique  system,  yet  in  contractual  terms  SPS 
remained  a  licensed  commercial  product  and  DoD  was  locked  into  a 
sole  source  for  life  cycle  support; 

-  initial  users  were  complaining  about  the  system's  limited 
capability,  the  newly  installed  SPS  software  was  not  being  used 
at  13  of  25  sites  that  we  visited,  and  the  "equal  or  better 
functionality"  guidelines  had  not  been  enforced; 

-  the  schedule  was  slipping  and  costs  had  increased; 

-  training,  guidance  and  help  desk  support  for  users  were 
inadequate ; 

-  inefficient  workarounds  were  frequently  in  use  at  sites 
where  SPS  had  been  deployed; 


-  a  more  accurate  life-cycle  cost  estimate  was  needed;  and 
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-  although  considerable  time  had  passed  since  Congress 
provided  additional  contracting  flexibility  for  buying 
commercial  products,  DoD  still  lacked  internal  guidance  on 
acquiring  commercial  computer  software  for  major  information 
systems,  including  the  SPS. 

The  management  responses  to  our  May  1999  report  were  mixed.  The 
Assistant  Secretary  of  Defense  (Command,  Control,  Communications 
and  Intelligence)  agreed  to  issue  guidance  on  acquiring 
commercial  software  and  did  so  in  July  2000.  During  the  audit, 
in  May  1998,  a  Requirements  Board  was  established  to  evaluate 
deficiencies  identified  by  users.  The  Board  determined  that  36 
additional  capabilities  were  needed.  The  program  office  also 
took  various  measures  to  improve  customer  support.  By  and 
large,  however,  program  office  managers  appeared  to  believe  that 
the  problems  cited  in  our  report  had  been  solved  in  the  latest 
releases  of  the  SPS  software.  As  subsequent  events  proved,  this 
was  not  the  case . 

By  December  1999,  four  versions  of  SPS  had  been  deployed,  the 
latest  being  version  4.1.  At  about  that  time,  the  House 
Committee  on  the  Budget  opened  its  own  Internet  hotline  for 
citizens  to  report  waste  in  Government  and  received  multiple 
complaints  about  SPS  from  Defense  personnel  at  sites  that  had 
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received  it.  The  Committee  referred  these  allegations  to  us  and 
we  decided  to  conduct  a  web-based  survey  of  statistically 
selected  SPS  version  4.1  users.  Although  the  Department  had 
great  difficulty  providing  accurate  user  identities  and  e-mail 
addresses,  we  received  about  600  replies  to  the  survey 
instrument  between  May  and  July  2000. 

The  survey  results  were  published  in  Audit  Report  No.  D-2001- 
075,  Standard  Procurement  System  Use  and  User  Satisfaction, 

March  13,  2001.  The  only  reasonably  good  news  was  that  86 
percent  of  the  SPS  users  stated  that  the  system  was  available 
always  or  most  of  the  time.  Otherwise,  although  numerous 
software  improvements  had  been  made  and  many  respondents  praised 
the  system's  potential,  user  dissatisfaction  levels  were  still 
unexpectedly  high.  For  example: 

•  61  percent  of  SPS  users  preferred  a  procurement  system 
other  than  SPS, 

•  46  percent  of  the  users  stated  that  the  number  of 
workarounds  had  increased, 

•  51  percent  of  the  users  stated  that  productivity  had  not 
increased  since  SPS  version  4.1  was  implemented,  and 
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•  64  percent  of  the  users  stated  that  SPS  had  not 

substantially  contributed  to  the  DoD  goal  of  paperless 
contracting. 

Further,  based  on  survey  responses,  we  projected  that  about  27 
percent  of  the  personnel  licensed  to  use  SPS  version  4 . 1  had  not 
used  it,  because  SPS  either  lacked  the  functionality  for  those 
sites  or  employees  received  SPS  when  it  was  not  needed  to 
perform  their  jobs.  We  estimate  that  the  Department  spent  up  to 
$2.1  million  on  licenses  for  users  who  could  not  or  did  not  need 
to  use  SPS. 


We  made  numerous  recommendations  and  offered  the  following 
general  conclusions . 


"DoD  has  experienced  a  50  percent  reduction  in  the 
procurement  workforce  without  a  commensurate  reduction  in 
workload.  Conceptually,  SPS  should  assist  in  automating 
and  standardizing  a  variety  of  procurement  tasks  and  thus 
assist  in  more  efficiently  completing  the  workload. 
According  to  the  survey,  however,  functionality  remains  a 
serious  concern.  Management  needs  to  respond  to  this 
concern  when  deploying  new  SPS  versions  and,  if  SPS  does 
not  fully  meet  mission  needs,  should  consider  supplementary 
and  alternative  tools  for  the  procurement  workforce." 


"There  is  a  need  for  more  appropriate  testing  prior  to 
future  deployment.  About  38  percent  of  respondents  contend 
that  SPS  version  4.1  had  only  some  or  none  of  the 
functionality  needed,  despite  testing.  Present  performance 
measures  do  not  address  mission  needs  such  as  enhancing 


customer  service,  reducing  problem  disbursements, 
increasing  contracting  personnel  productivity,  or 
eliminating  redundancy." 
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Several  actions  have  been  taken,  in  response  to  our  audits  and 
various  other  reviews,  which  lead  me  to  be  cautiously  optimistic 
about  the  future  of  this  effort.  For  example: 

-  the  Director,  Defense  Procurement,  explicitly  directed 
the  Military  Departments  and  Defense  Agencies  to  assess  and 
validate  functional  requirements  against  user  needs  before 
deploying  any  future  versions  of  the  SPS; 

-  it  is  apparent  that  the  Department  insisted  on  rigorous 
testing  of  SPS  version  4.2,  which  was  actually  returned  to  the 
contractor  for  rework  last  year; 

-  a  contractor  has  been  hired  to  provide  Independent 
Validation  and  Verification  of  SPS  software; 

-  independent  management  assessments  by  consultants  and  DoD 
acquisition  experts  strongly  indicated  higher  than  acceptable 
levels  of  risk  remaining  in  the  program  and  seem  to  have  added 
impetus  to  efforts  to  improve  its  discipline  and  risk 


management ; 
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-  a  working  group  is  developing  the  performance  measures 
that  are  badly  needed  to  monitor  this  program  properly; 

-  the  SPS  acquisition  strategy  and  economic  analysis  will 
be  reviewed  and  updated;  and 

-  senior  Office  of  the  Secretary  of  Defense  managers  are 
clearly  more  engaged  in  this  program's  issues  than  was  the  case 
earlier  in  the  program. 

In  conclusion,  I  believe  that  the  SPS  concept  is  fundamentally 
sound  and  the  Department  will  be  well  served  by  replacing  the 
outmoded  and  unintegrated  legacy  systems.  Nevertheless,  despite 
the  numerous  commendable  actions  recently  taken,  it  would  be 
premature  to  assume  that  further  execution  of  this  program  will 
necessarily  be  smooth.  In  the  near  term,  the  program  continues 
to  need  close  attention  and  perhaps  more  restructuring  if  it  is 
to  fulfill  the  Department's  expectations.  The  exact  scope  of 
the  program  and  its  funding  requirements  need  to  be  resolved 
this  year.  From  the  standpoint  of  DoD  financial  management,  the 
effectiveness  of  the  data  transfer  between  SPS  and  the  DoD 
finance  and  logistics  systems  will  be  an  especially  significant 


concern,  particularly  because  those  other  systems  and  related 
standards  are  in  various  stages  of  redesign  or  replacement. 


12 


The  recent  suggestion  by  GAO  that  a  DoD  "lessons  learned"  report 
be  written  on  the  SPS  experience  is  a  very  good  one,  although  it 
would  be  important  to  ensure  that  such  a  report  be  thoroughly 
objective  and  reflect  what  may  be  a  range  of  opinion  about  what 
those  lessons  are. 

Thank  you  again  for  considering  these  views . 


